Real-time geometric lens distortion correction using a graphics processing unit

Optical imaging systems often suffer from distortion artifacts which impose important limitations on the direct interpretation of the images. It is possible to correct for these aberrations through image processing, but due to their calculation-intensive nature, the required corrections are typically performed offline. However, with image-based applications that operate interactively, real-time correction of geometric distortion artifacts can be vital. We propose a new method to generate undistorted images by implementing the required distortion correction algorithm on a commercial graphics processing unit (GPU), distributing the necessary calculations to many stream processors that operate in parallel. The proposed technique is not limited to affine lens distortions but allows for the correction of arbitrary geometric image distortion artifacts through individual pixel resampling at display rates of more than 30 frames per second for fully processed images (1024 × 768 pixels). Our method enables real-time GPU-based geometric lens distortion correction without the need for additional digital image processing hardware. 1 Introduction Optical lenses can introduce a wide variety of geometric distortion artifacts to an imaging system, which reduces the perceptive quality of the resulting images. Especially in imaging systems with a wide viewing angle, such as endoscopes, image distortion can be very pronounced. Local variations in the index of refraction of the medium through which an object is observed will also cause image distortions. Such distortions cannot be described by a simple affine transformation. To correct for these visual aberrations, numerous recalibration algorithms have been described 1,2 and software that remaps the produced image pixels accordingly is widely available. 3 The recalibration process typically requires more processing power than modern central processing units (CPUs) can deliver when applied in real-time, and the lens distortion correction process is therefore often performed offline. However, in many applications in medical imaging, computer vision, and industrial monitoring, interaction with the user or environment requires real-time production of undistorted images to enable the accurate perception of distance and shape. 4 To this end, a distortion correction method was described 5 that effectively exploits the capability of standard graphics processing units (GPUs) to reposition triangle vertex coordinates at high speeds. This method can only correct for the dominant radial distortion effects (also known as barrel or pincushion distortion) as it relies on a general affine transformation formula and does not permit pixel-by-pixel resampling. To enable pixel-by-pixel resampling at video rates, special-purpose hardware-based solutions based on field programmable gate arrays (FPGAs)